01 Prompt engineering

戦略 (Strategies)

ref: GPT best practices

Start with

clear instructions

split into simpler subtasks

time to "think"

test changes systematically

一度にいくつも変更してもぐらたたきにしない

evalセットとLLMOpsが必要

Extend to

provide reference text

external tools

good for（適する）

testingとlearningが早い

ベースライン

プロンプトエンジニアリングで始め、タスクによってはプロンプトエンジニアリングで完結する

not good for（適さない）

新しい情報を導入できない

複雑なスタイルやメソッドは複製できない

トークン使用量を最小化できない

最初にはいいが、特定の問題に対処するgreatな方法ではない

ref: https://youtu.be/ahnGLM-RC1Y?si=7SdP7XiQnGPSQ3S7&t=510

曖昧の指示のプロンプト例とclearな例

ステップごとにアプローチさせる

考える時間、ReActフレームワークへの言及

3つやっているプロンプト例（clear、time to think、breakdown）

code:prompt_example

System

Clear instructions

You will be presented with a news article. Your task is to identify any opinions expressed about the government, and their sentiment.

Give time to think

Approach this task step-by-step, take your time and do not skip steps:

Break down complex tasks

1. Read a paragraph of the news article

2. Determine whether an opinion is expressed in this paragraph, if not, continue to the next paragraph

3. if there is an opinion, extract a JSON with keys as follows:

* opinion: allowable values are "positive", "negative" or "neutral"

* evidence: contains a list of strings evidencing the opinion

* speaker: the person or government body who expressed the opinion

4. Include as much evidence as possible, even if an opinion has already been stated

<2パラグラフ与えている>

（プロンプトの）どのトークンがモデルに影響を与えているかは不明

-> few shot（入出力のペアを追加）

code:few_shot_example

System

You are an assistant responsible for helping detect whether the retrieved document is relevant to the query. For a given input, you need to output a single token: "Yes" or "No" indicating the retrieved document is relevant to the query.

User

Provide few-shot examples

Query: How to plant a tree?

Retrieved document: Cars were invented in 1886, when German inventor Carl Benz patented his Benz Patent-Motorwagen.345 Cars became widely available during the 20th century. One of the first cars affordable by the masses was the 1908 Model T, an American car manufactured by the Ford Motor Company. Cars were rapidly adopted in the US, where they replaced horse-drawn carriages.6 In Europe and other parts of the world, demand for automobiles did not increase until after World War I.7 The car is considered an essential part of the developed economy.

Relevant to the query?

Assistant

User

Query: Who is Bill Clinton?

Retrieved document: William Jefferson Clinton is an American politician who served as the 42nd President of the United States from 1993 to 2001.

Relevant to the query?

Assistant

Yes